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A  PROPOSED  METHODOLOGY  FOR  THE  SPATIAL 


CHARACTERISATION  OF  FOLIAGE  BACKGROUNDS 


1 .  INTRODUCTION 


The  procedure  outlined  in  the  following  Note  is  proposed  as  a  means 
of  quantifying  visual  texture  perception  of  images  of  natural  foliage.  The 
work  is  in  response  to  Army  Task  ARM  81/139  (subsequently  replaced  by  ARM 
84/169),  which  required  a  methodology  for  such  characterisation  to  be 
developed,  current  work  is  oriented  towards  ground-based  observation  of 
natural  foliage,  with  the  use  of  black-and-white  photographic  negatives  as  the 
primary  image  source.  Emphasis  in  the  work  will  be  placed  on  the  relation 
between  visual  perception  of  projected  images  of  foliage  scenery  and  digital 
encodings  of  the  corresponding  images. 


In  outlining  the  procedure  through  sections  2  and  3,  colour  is  used 
in  its  general  sense,  subsuming  monochromatic  images  as  a  subset  of  colour 
images,  and  texture  is  likewise  treated  as  arising  from  both  brightness  and 
chroma  variations.  Although  the  published  literature  on  texture  deals  only 
with  monochromatic  systems,  the  concept  cannot  reasonably  be  restricted  to 
such  images,  and  hence,  where  it  is  reasonable  to  do  so  here,  the  more  general 
sense  of  texture  is  used.  However,  in  elaborating  on  mathematical  descriptors 
of  texture  in  section  4,  discussion  is  restricted  to  monochromatic  textures 
because  of  both  limitations  in  the  available  hardware,  and  the  assumed  greater 
simplicity  of  working  in  only  the  brightness  dimension  of  colour.  It  is, 
however,  assumed  that  analogous  procedures  could  be  applied  to  full  colour 
textures  -  though  at  some  considerably  greater  degree  of  complexity. 


The  widespread  trend  in  military  camouflage  towards  patterning  of 
nets,  uniforms,  vehicles  and  aircraft  has  relied  on  patterns  developed  almost 
entirely  by  heuristic  methods  supported  by  limited  post  facto  evaluation.  By 
quantifying  visual  texture  in  a  way  related  to  the  perception  of  texture 
differences,  a  basis  for  more  systematic  pattern  development  will  be 
provided.  In  particular  the  problem  of  determining  a  compromise  pattern  over 
a  range  of  foliage  textures  may  be  systematically  approached  and  the  basis  for 
any  recommendation  for  future  camouflage  patterning  more  fully  justified.  In 
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addition,  as  the  need  for  wider  spectral  range  countermeasures  arises,  a 
fuller  understanding  of  patterning  in  non-visual  spectral  ranges  may  be 
required.  A  further  consideration  in  developing  a  quantification  of  visual 
texture  based  on  human  perception  is  its  potential  use  in  synthetic  texture 
generation,  both  for  training  simulators  and  for  preprocessing  of  displayed 
images  produced  by  future  generations  of  surveillance  instruments. 

Patterning  can  be  described  as  the  superposing  on  an  object  of  fine 
detail  colour  variations  about  a  colorimetric  mean,  with  an  analogous 
definition  for  non-visual  spectral  images.  In  camouflage  patterning  the 
intended  function  is  to  produce  colour  variations  within  the  target  perimeter 
which 

(i)  are  colorimetrically  and  spatially  consistent  with  those  in  the 
backgrounds  within  which  the  target  operates; 
or 

(ii)  reduce  the  detectability  of  known  visual  cues,  such  as  horizontal 
edges  and  shadows; 


or  both  (i)  and  (ii)  above. 

The  first  function  relates  closely  to  patterning  for  variable  shape 
targets  such  as  nets  and  uniforms,  and  its  application  is  detailed  in  Skinner 
and  Jenkins  [11.  The  second  is  more  appropriate  for  fixed  shape  targets  such 
as  aircraft,  ships,  vehicles,  and  buildings.  Beckwith  12)  describes  an 
approach  to  vehicle  patterning  based  primarily  on  this  latter  function. 


2.  QUANTIFICATION  OF  TEXTURE 


The  concept  of  texture*  is  a  psychophysical  construct  used  to 
describe  those  spatial  variations  in  colour  which  give  rise  to  a  perceived 
local  uniformity  despite  being  individually  resolvable,  or  nearly  so. 
Typically  such  variations  are  rapid  by  comparison  with  changes  in  features  of 
the  image  which  are  perceived  in  some  structural  relationship  to  each  other. 
For  example,  a  scene  showing  a  section  of  a  golf  course  in  which  stand 
isolated  trees  would  have  a  background  texture  of  grass,  with  trees  and 
bunkers  being  some  of  the  features. 


I  will  not  attempt  here  a  rigorous  delineation  between  textures  and 
features,  but  suggest  that  numeroslty  of  resolvable  elements  is  a  critical 
feature  Ill  -  13),  as  well  as  angular  subtense  of  resolvable  elements 
l 14),  and  edge  contrast. 
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Texture  quantification,  therefore,  requires  at  least  a  measure  of 
the  (spatial)  rapidity  and  extent  of  colour  variation  from  the  local  mean,  and 
may  also  need  measures  of  spatial  correlation,  or  other  structural  features  of 
the  variation.  Various  descriptors  have  appeared  in  the  image  processing 
literature,  some  based  on  Fourier  analysis  [3],  others  on  stochastic  processes 
(4-61,  and  others  on  structural  analysis  approaches  (7-8).  These  descriptors 
have  often  arisen  from  an  intuitive  approach  to  specifying  those  aspects  of 
texture  which  are  perceptually  significant.  From  this  approach,  concepts  such 
as  coarseness,  busyness,  regularity,  line-likeness,  etc.  have  arisen. 


A  significant  attempt  to  systematize  the  relationship  between 
perception  of  texture  and  mathematical  descriptors  was  presented  by  Tamura  et 
al.  (91,  who  used  a  set  of  textures  from  the  Brodatz  album  (10i.  They 
verbally  defined  six  characteristics  of  texture,  then  presented  all  possible 
pairwise  combinations  of  the  textures  to  the  subjects,  who  were  to  specify 
which  of  the  pair  was  higher  on  each  of  the  characteristics  described.  These 
characteristics  were  labelled  coarseness,  contrast,  directionality,  line- 
likeness,  regularity,  and  roughness.  A  set  of  mathematical  descriptors  was 
then  developed  which  were  considered  appropriate  for  quantifying  the 
characteristics  described.  For  these  measures  the  resulting  rank  correlation 
between  the  mathematical  descriptors  and  the  averaged  subjective  ratings  on 
that  characteristic  ranged  from  0.615  to  0.904.  There  was  considerable 
intercorrelation  of  the  various  dimensions  in  both  the  subjective  and  the 
mathematical  measures. 


These  results  of  Tamura  et  al.  provide  justification  for  the  use  of 
terms  such  as  coarseness,  directionality,  etc.  However  they  do  not  provide  a 
basis  for  the  development  of  a  psychophysics  of  texture  having  a  value  similar 
to  that  of  the  psychophysics  of  colour.  This  is  because  the  dimensions  used 
are  strongly  correlated  and  no  attempt  has  been  made  to  search  for  a  better 
definition  of  the  perceptual  dimensions.  Further,  there  is  no  evidence  that 
estimates  of  texture  difference  made  along  individual  specified  dimensions  are 
valid  when  judging  overall  texture  differences  simultaneously  along  all 
available  dimensions,  statistical  techniques  such  as  factor  analysis,  cluster 
analysis  and  multidimensional  scaling  allow  one  to  search  systematically  for 
such  perceptual  dimensions. 


A  partial  validation  of  the  use  of  multidimensional  scaling  in 
extracting  the  perceptual  dimensions  of  colour  was  provided  by  Wright  (is), 
who  showed  that  perceptual  judgements  on  different  coloured  equally  luminous 
reflectance  tiles  gave  results  consistent  with  two  major  dimensions  of  colour 
difference,  these  two  dimensions  corresponding  to  hue  and  saturation.  More 
recently  Sokolov  et  al.  (16)  have  used  multidimensional  scaling  to  extract 
three  dimensions  underlying  the  perception  of  colour.  These  authors  found  the 
same  underlying  structure  for  both  a  colour  designation  technique  and  a  paired 
comparison  technique.  Harvey  and  Gervals  (17),  in  a  study  of  texture 
discrimination  using  synthetic  one  dimensional  textures,  used  multidimensional 
scaling  to  show  that  texture  discrimination  was  well  defined  by  three 
perceptual  dimensions  corresponding  to  three  spatial  frequency  bands.  Thus 
multidimensional  scaling  appears  to  be  an  appropriate  tool  to  use  in 
determining  the  dimensions  of  perceptual  spaces. 
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The  method  proposed  here  mafces  no  assumptions  about  which  aspects  of 
texture  might  be  important.  Rather  it  uses  judgements  regarding  texture 
dissimilarity  to  provide  proximity  (distance)  measures  on  a  set  of  textures, 
from  which  an  n-dimensional  configuration  of  points  -  each  point  representing 
a  texture  -  is  determined,  this  configuration  being  consistent  with  their 
judged  proximities.  Following  the  determination  of  the  dimensions  underlying 
the  perception  of  texture  differences,  mathematical  descriptors  can  be 
developed  for  each  dimension. 


3.  MEASUREMENT  OF  PERCEIVED  TEXTURE  SIMILARITY 


Suppose  a  set  of  N  texture  samples  representing  the  textures  of 
interest  is  selected.  A  measure  of  judged  proximity  of  these  textures  is 
required.  This  is  obtained  using  the  method  of  paired  comparisons,  the 
comparisons  in  this  case  being  of  texture  differences.  Subjects  are  shown 
three  textures  side-by-side,  with  the  left  and  right  hand  textures  being 
judged  for  degree  of  similarity  to  the  central  texture  (see  Fig.  l).  Thus  two 
texture  differences  are  being  compared  -  between  the  left  texture  and  central 
texture,  and  between  the  right  texture  and  central  texture.  Each  texture  is 
placed  in  the  centre  and  all  possible  pairs  of  textures  are  compared  to  the 
central  texture.  Because  the  textures  are  from  natural  foliage  scenes  some 
non-uniformities  exist  within  a  particular  texture  which  may  lead  to  an 
assymetric  assessment  of  similarity  according  to  whether  it  is  to  the  left  or 
right.  Hence  each  texture  appears  on  both  sides  of  the  central  texture. 

subjects  are  required  to  judge  which  of  the  two  outer  textures  is 
more  similar  to  the  central  one,  with  the  option  of  judged  equality,  scoring 
is  -l,  0,  +1  according  to  whether  the  judgement  is  "left-hand”,  "equally 
similar",  or  "right-hand"  respectively.  Given  N  textures,  and  allowing  for 
identical  textures  to  be  placed  simultaneously  in  two  or  three  locations, 
there  are  N3  judgements  to  be  made.  From  these  an  N  x  N  dissimilarity  matrix 
for  the  textures  can  be  derived,  as  described  in  the  Appendix. 

The  exposure  duration  for  a  single  comparison  set  of  three  textures 
is  chosen  by  consideration  of  the  requirements  of  the  comparison  task.  It  is 
assumed  that  a  series  of  fixations  is  made,  with  at  least  one  fixation  on  each 
of  the  outer  two  textures  and  two  on  the  central  texture.  Based  on  reaction 
latencies  obtained  in  other  texture  discrimination  experiments  (ill,  and  also 
in  studies  of  visual  comparison  processes  del,  a  time  of  approximately  o.5s 
per  fixation  is  estimated  for  this  tasK.  Any  longer  exposure  would  probably 
enhance  individual  differences  in  processing  between  what  Cunningham  et  al. 
(181  describe  as  holistic  versus  analytic  processors.  It  is  the  holistic 
processing  which  is  of  interest  here.  Hence  an  exposure  time  of  2.0 s  is 
proposed  for  each  comparison  set. 

In  order  to  obtain  a  useful  relation  between  the  digital  image  and 
results  from  an  experiment  of  the  type  described  here,  it  is  imperative  that 
the  function  mapping  the  grey  levels  onto  perceived  image  brightness  be  well- 
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defined.  Probably  the  most  suitable  procedure  is  to  determine  the  functional 
relation  between  grey  levels  and  display  brightness  for  the  specific  hardware 
used,  and  then  transform  the  raw  image  prior  to  display  such  that  equal  grey 
level  differences  give  equal  brightness  differences  for  all  grey  levels  and 
field  positions  used.  Mathematical  descriptors  can  then  be  derived  from  grey 
levels  of  the  (unmodified)  raw  image.  The  image  modification  matrix  is 
obviously  dependent  on  the  particular  system  used,  but  any  results  obtained 
will  be  system-independent. 


4.  MATHEMATICAL  DESCRIPTORS  OF  TEXTURE  SIMILARITY 


In  this  and  the  following  sections  attention  is  confined  to 
monochromatic  textures,  though  there  is  a  need  for  future  worlc  to  develop 
procedures  applicable  to  full  colour  images. 

Digitisation  of  (monochrome)  images  recorded  on  photographic  film 
normally  involves  quantizing  the  transmittance  of  a  matrix  of  small  regions  of 
the  photographic  image  into  equally-spaced  steps.  This  quantization  is 
commonly  done  by  a  special  purpose  TV  camera,  or  -  for  higher  spatial 
resolution  -  by  a  rotating  drum  scanner.  The  result  is  to  produce  a  matrix  of 
transmittance  values  with  arbitrary  zero  and  scale  (determined,  for  instance, 
by  the  "black  level"  and  “gain"  settings  of  a  TV  camera),  commonly  the  matrix 
will  be  square  of  si2e  512  x  512  with  matrix  elements  lying  in  the  range  0  to 
255,  corresponding  to  an  8  bit  binary  representation.  Each  picture  element 
sampled  is  called  a  pixel,  and  each  matrix  element  representing  the  luminance 
of  a  pixel  is  called  a  grey  level. 


Suppose  a  segment  of  "uniform"  texture  is  digitised  and  the  discrete 
two-dimensional  Fourier  transform  of  the  the  matrix  of  grey  levels  found. 

This  gives  a  set  of  (generally  complex)  values,  F^j,  i  =  0,1,... M-l,  j  = 
o,i,... M-l  of  the  spatial  frequency  function.  A  subset  of  these 
values  {F  }  is  selected,  from  which  a  summary  measure,  s(F,m),  of  texture  is 
derived  such  that  s(F,m)  correlates  strongly  with  the  mth  dimension  of 
perceived  texture  similarity  as  determined  by  the  earlier  psychophysical 
experiment.  The  simplest  form  of  s(F,m)  is  a  linear  sum 


S(F,m> 


L 

=  E 

l=i 


a  F 
ml  ml 


where  the  a  are  weighting  coefficients.  s(F,n)  is  an  example  of  a 
mathematical  descriptor  of  texture,  in  this  particular  case  being  a  Fourier- 
based  descriptor  for  the  mth  perceptual  dimension,  corresponding  descriptors 
for  the  other  dimensions,  previously  determined  in  the  psychophysical 
experiment,  are  also  derived,  thus  producing  a  set  of  texture  measures  along 
the  known  perceptual  dimensions.  Any  texture  is  then  located  in  texture  space 
by  the  set  of  values  <s(F,i),  i=l,2,...D>  where  D  is  the  number  of  dimensions 
in  texture-perception  space.  The  similarity  of  two  textures  is  then 
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determined  by  simply  calculating  the  distance  between  them,  using  the  metric 
specified  in  the  multidimensional  scaling  analysis. 

The  previous  paragraph  used  Fourier-based  descriptors  as  an  example 
of  one  type  of  mathematical  descriptor.  Haraliclc  (191  has  proposed  the  use  of 
co-occurrence  (or  spatial  grey  level  dependence)  matrices  for  texture 
analysis.  A  co-occurrence  matrix  for  a  given  vector  displacement,  d,  has 
elements  such  that  is  the  observed  relative_f requency  that,  for  a 
pixel  of  grey  level  i,  the  pixel  at  a  displacement  +  d  is  of  grey  level  j.  A 
set  of  descriptors  {S  .}  based  on  co-occurrence  matrices  could  be  sought. 

Laws  (5 1  has  proposed  a  set  of  pixel  operators  which  have  proven  to  be 
powerful  discriminators  of  texture  (201,  and  a  set  of  texture  descriptors 
based  on  these  may  also  be  valuable. 


In  searching  for  appropriate  descriptors,  both  the  Known 
characteristics  of  human  perception  of  texture  and  the  limitations  of  the 
image  acquisition  and  display  system  need  to  be  considered.  Julesz  1 2 1 J ,  and 
Julesz  et  al.  (22)  have  postulated  that,  to  a  very  considerable  extent,  humans 
are  unable  to  differentiate  textures  differing  only  in  their  third  and  higher 
order  statistics,  but  can  spontaneously  discriminate  those  differing  in  their 
second-order  statistics.  It  is  appropriate  therefore  to  concentrate  on 
descriptors  which  are  based  on  second-order  statistics.  Descriptors  mentioned 
in  the  previous  paragraph  all  fall  within  this  category.  Wilson  and  Bergen 
[231,  and  Bergen  et  al .  (24)  presented  evidence  for  human  static  visual 
perception  being  based  on  a  set  of  four  spatial  frequency  bands  whose 
frequency  varies  with  retinal  eccentricity.  This  approach  was  subsequently 
developed  by  Marr  (25),  and  also  investigated  by  Harvey  and  Gervais  (17)  who 
found  supporting  evidence  from  studies  of  the  perception  of  one-dimensional 
textures  synthesized  from  sinusoids  having  frequencies  in  various  of  these 
bands.  Caelli  (26)  has  developed  procedures  for  investigating  two-dimensional 
textures,  and  presented  evidence  for  orientationally-selective  channels 
having  a  variance,  o,  which  decreases  with  frequency,  »,  according  to 
o(»)  -  5  -  0.125  •.  Qualitatively  similar  results  were  obtained  by  Phillips 

and  Wilson  (27),  though  they  found  a  more  rapid  decrease  in  half-width  with 
spatial  frequency. 


All  of  the  above  results  indicate  that  a  reasonably  limited  set  of 
quantities  based  on  spacing  of  pixels  and  selected  orientations  can  be 
expected  to  describe  the  dimensions  of  texture  perception  of  natural  textures. 


In  deriving  a  particular  descriptor,  the  selection  of  weighting 
factors,  a  ,  is  based  on  maximizing  the  correlation  between  judged  texture 
similarity  dimension  and  the  calculated  value  of  the  corresponding  texture 
descriptor.  This  maximization  can  be  achieved  using  the  statistical  technique 
of  canonical  correlation. 
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CONCLUSION 


5  . 


The  methodology  as  outlined  is  applicable  to  monochromatic, 
continuous-tone  natural  textures  presented  in  two  dimensions,  with 
appropriate  development  of  descriptors  it  should  be  applicable  to  coloured 

natural  textures. 
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APPENDIX 


DERIVATION  OF  A  DISSIMILARITY  MATRIX  FOR  TEXTURES 


Suppose  N  =  5  textures  are  to  be  judged  for  similarity.  For  each 
subject  a  matrix  of  judgements  is  produced  for  each  reference  texture.  For 
example,  with  texture  4  as  the  reference  (central)  sample,  the  matrix  could 
be  as  follows 


Right 

l 

hand 

2 

texture 

3 

number 

4  5 

row  sums 

Left 

1 

0 

-1 

1 

1 

1 

2 

hand 

2 

l 

0 

l 

1 

1 

4 

texture 

J 

-l 

-1 

0 

1 

-1 

-2 

number 

4 

-l 

-1 

-l 

0 

-1 

-4 

5 

0 

-1 

-l 

1 

0 

-1 

Column  sums 

-l 

-4 

0 

4 

0 

It  should  be  noted  that  the  table  indicates  two  apparently  inconsistent 
comparisons 


(i)  when  the  judgement  involves  textures  3  and  5  simultaneously 
whichever  is  on  the  left  appears  more  similar  to  texture  4; 

(ii)  when  textures  5  and  1  appear  simultaneously  no  discrimination  is 

made  when  5  is  on  the  left,  but  when  it  is  on  the  right  it  is  judged 
more  similar  to  4  than  is  texture  l . 


Clearly,  the  more  closely  a  texture  resembles  the  central  texture, 
the  more  the  corresponding  row  will  have  negative  numbers  and  the 
corresponding  column  will  have  positive  numbers.  Hence,  if  we  define  a 
function  for  each  non-central  texture  j  (j  -  i,2,...n)  as  the  row  j  sum  minus 
the  column  j  sum,  this  will,  given  perfect  consistency  in  judgement,  range 
from  —2 ( N-l )  for  the  most  similar  texture  (which  is  the  same  as  the  central 
texture),  to  2<N-l)  for  the  least  similar.  In  our  example  this  gives 
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Texture  number 


1  2  3  4  5 

3  8-2-8  -1 


Subtracting  the  most  negative  value  from  all  gives  a  dissimilarity  score  which 
is  zero  for  the  most  similar  texture,  with  larger  values  indicating  less 
similar  textures.  These  values  provide  one  column  in  an  individual’s 
dissimilarity  matrix.  By  repeating  the  process  for  all  reference  textures  a 
complete  dissimilarity  matrix  is  obtained  for  each  subject.  The  average 
matrix  for  each  subject  or  across  all  subjects  may  provide  the  input  to  a 
multidimensional  scaling  program.  It  is  expected  that  the  input  matrix  would 
have  zero  or  near-zero  diagonal  elements,  but  it  is  not  expected  that  it 
should  be  symmetrical  about  the  diagonal,  since  all  columns  are  individually 
scaled.  However,  multidimensional  analysis  can  be  carried  out  using  only  in¬ 
column  comparisons. 
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